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ABSTRACT 



This report provides an introduction to the use of opinion surveys 
in gaining information about a school system in order to make decisions. 
The following topics are addressed: 



(1) 


Reasons for surveying; 


(2) 


Selecting the respondents; 


(3) 


Survey methods; 


(A) 


Selecting the survey sample; 


(5) 


Structuring the survey; 


(6) 


Writing the questions; 


(7) 


Training Interviewers; 


(8) 


Pre-testlng the survey; 


(9) 


Analyzing the results; and 


(10) 


Reporting the results. 



This information should be helpful to a school administrator or 
school board member in the process of considering the use of an 
opinion survey* 



Why Survey? 

School boards and school administrators need current, accurate informa- 
tion in order to make decisions. In general, the information sources of 
school systems are not systematic. People with complaints are more likely 
to voice their opinions than those who are satisfied, two people with the 
same type of complaint may contact different people connected with the schools, 
and the same type of situation may cause one person to offer praise or 
criticism to the school district while another person might remain silent. 

An opinion survey is one means of filling some of the comniinication 
needs of a school district. Two types of information can be collected: 

(1) Attitudes about important community concerns; and 

(2) Data on community understanding of school programs or 
community information sources about the schools. 

In addition to information collection, a survey may be used to include 

board-based participation in decision-making. This is an especially important 

use in view of the new Pennsylvania Department of Education's Long-Range 

Planning guidelines which stress community involvement in the planning process. 

Selecting the Respondents 

While the major constituents of a school district are parents of school 
children or taxpayers, pupils and school employees are also possible target 
groups for opinion surveys. The group or groups selected depend(s) upon 
the reasons for the survey, i.e., taxpayers may be questioned concerning 
financial plans, parents surveyed about their satisfaction with school 
programs, or a wide range of people might be asked about a plan with wide 
impact such as year-round schools. 

6 
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Survey Method 

There are three methods of conducting surveys : 

(1) Mail questionnaires; 

(2) Telephone interviews; and 

(3) Face-to-face interviews. 

Table I summarizes the features of each of these survey methods. 



TABLE I: CHARACTERISTICS OF SURVEY METHODS 
SURVEYING FACTORS 



METHODS 


RESPONSE 
RATE 


COST 


POSSIBLE 
RESPONSE 
BIAS 


OPPORTUNITY 
FOR PROBING 


Mall 


Low-Moderate 


Moderate 


Very low 


Very little 


Telephone 


High 


Expensive 


Moderate 


High 


Face-to-face 


High 


Expensive 


High 


High 
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Each of these factors will vary in importance depending upon the purpose 
of the survey. The response rate is the percentage of people answering 
the survey questions out of the total number of people asked. A high response 
rate is important. For example, if only half of the people asked to respond 
to a survey do respond, the attitudes of these people may or may not. be 
in accordance with the attitudes of those people not responding. Generally, 
if the response rate is less than 80%, it will be impossible 
to determine whether the results are representative of the population as a 
whole, or merely representative of those members interested enough to respond. 
Survey methods involving personal contact are more likely to get someone to 
respond to the questions, although return envelopes and follow-up contacts 
can increase response rates for mail questionnaires. Telephone and face-to-face 
Interviews are naturally more expensive because they are labor intensive. 
Even if volunteers are used, they must still be trained to conduct interviews, 
and the use of volunteers may increase resjjonse bias. 
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Response bias refers to the degree to which a survey records a 
distortion rather than a person's "true opinion." It usually occurs 
because respondents to a survey try to answer questions the way they think 
the survey sponsor wants them to answer. For example, "Don't you think 
the school board is doing a good job?" is a biased question because it is 
slanted toward obtaining a positive response. In a face-to-face survey, 
the interviewer's facial expressions, verbal responses, etc. to the 
respondent's answer to one question could have a detrimental influence on 
later questions. That is, the respondent may form answers to please or 
displease the interviewer which, of course, results in a biased response. 
All but experienced interviewers, therefore, are likely to cause bias 
in a personal interview. Telephone surveys are less likely to have bias 
than face-to-face interviews. Written questionnaires are most likely 
to obtain candid, unbiased answers from respondents because they allow 
the respondent the greatest degree of anonymity. 

The characterizations of each of these survey methods are generaliza- 
tions. Any method will yield poor results if poorly carried out. For 
example, some mail surveys may have higher response rates than a telephone 
survey if interviewers call at inappropriate times; or a skilled interviewer 
In a face-to-face interview may get less biased answers than a poorly 
worded mail questionnaire. 

Selecting the Survey Sample 

In conducting opinion surveys, resource limitations usually dictate 
that only a portion of the total group of interest be polled. This is 
certainly the case where the parents or citizens of a school district are 
the respondent group. Drawing a sample from a larger population involves: 

8 
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(a) identifying a population for which some information is sought; 

(b) examining some members of the population; and 

(c) extending the findings from the sample to the entire population. 

It is therefore most important that the sample selected is representa- 
tive of the whole group over a specified set of variables. Some of the variables 
which might be important in a school district survey of parents are: 

(a) sex, 
0>) age, 

(c) number of children, 

(d) years in school district, 

(e) school attendance area, 

(f) education level, and 

(g) renting or owning a home. 

Selecting a representative sample is facilitated by two activities: 

(1) choosing the sample in such a manner that it is likely to be 
representative; and 

(2) checking the sample to see if it is representative. 
Choosing the sample by a systematic or random method increases the 

likelihood that it will be representative. For example, suppose that a 
school district wants to select 500 of 5000 households to respond to an 
opinion survey. One type of systematic sampling method would involve choosing 
every 10th household from a list of households. Unless there is a bias in 
the way households are arranged on the master list, this method should provide 
a representative sample. In this type of random sampling, each household 
has a 10% chance of being selected. A random number table or a computer could 
then be used to "decide" which 500 of the 5000 households should be selected. 

9 
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Once the sample is chosen it can be compared to the whole group to 
see if it is representative with respect to the important variables. Statistical 
tests exist to determine whether the sample is within acceptable limits 
of the whole group on each variable.* Given a confidence level (usually 
90Z or 95%), these tests can be used to determine such questions as: 

(a) Is the average number of children per family "the same" for the 
sample and the whole group? ; and 

(b) Is the proportion of households in Franklin School attendance 
area "the same" for the sample as for the whole group? 

If there is confidence that the sample was selected so that it was 
representative, it may not be necessary to make the effort to test whether 
it is representative prior to conducting the survey. However, this test 
should be performed on the respondent group before releasing the survey results. 
If the respondent group is not representative on one or more important 
variables, then either the results must be corrected (by weighting the number 
of responses) or the bias (or biases) should be reported with the survey 
results. Recent census figures provide a means of checking several 
variables for parents or citizens. 

A specialized form of random sampling, called a stratified random 
sample can be used to increase the likelihood of selecting a representative 
sample. In our previous example of selecting a sample of 500 from 5000 
households, let us assume that there are five elementary school attendance 
areas, each with 1000 households. If school attendance area is an important 
variable for this survey, then a stratified sample v;ould be selected by 
choosing 100 households from each of the five school attendance areas. In 



* The reader is referred to a statistics text for a discussion of these tests, 
For example, see Section 12.2, "Tests Tonceming Means" and Section 12.4.1, 

Tests Concerning Proportions" in Mathematical Statistics . John E. Freund, 
Prentice-Hall, Englewood Cliffs, New Jersey, 1962. 

o iO 
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effect, the stratified method guarantees that the sample will be representa- 
tive for each variable used in stratifying the sample. 

Another specialized form of sampling which may be used involves the use 
of clusters. For example, blocks or census tracts could be used as clusters 
and a sample of households could be drawn from each cluster rather than from 
the school district as a whole. Each cluster is assured to be homogeneous, 
so that each element of the cluster is equivalent to any other element In that 
cluster. In the case of cltjster sampling, special care should be exercised 
in checking the representativeness of the sample. 

A natural question, once the survey results are compiled, is "How 
precisely does the sample reflect the oninions of the whole grouTj?'-' 
Fortunately, the precision of a survey depends only upon the number of 
people responding and not the percentage of the population sampled, (assuming 
that the sample is representative). Table II provides estimates of the sampling 
error (or precision) for a question on an opinion survey. 



TABLE II** 
ALLOWANCE FOR SAMPLING ERROR 



Survey Question 


_100 


Numb'Jr of PeonlP pesDond^n^; 








Answer Z Near 10%, 90% 
Answer % Near 20%, 80% 
Answer % Near 30%, 70% 


7 
9 
10* 


5 
7 
8 


4 
5 
6 


bUU 

3 
4 

4 


750 
3 
4 
4 


1000 
2 
3 
4 


1500 
2 
2 
3 


Answer % Near 40%, 60% 


11 


8 


6 


5 


4 


4 


3 


Answer % Near 50% 


11 


8 


6 


5 


4 


4 


3 



**Adapted from Table 1, page 187, The Gallup Polls of Attitudes Towar d Education . 
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The table would be used in the following manner: Let us say that 
31% of a group of 100 respondents answered yes ^.o a particular question. There- 
fore, since the answer percentage is near 30% and the number of respondents 
is 100, we look at the starred figure in the table. This means that the 
31% obtained in the sample is subject to a 10% sampling error. Another 
way of stating this is that if the survey were repeated, 95 times out of 100*, 
the responses would be between 21% and^ 41% (31% ± 10%) answering yes to the 
same question. 

•# 

Several research methods books which offer more detailed descriptions 
of survey sampling have been included In the bibliography of this report 
for the reader's reference. 
Structuring the Survey 

Once the general reasons for undertaking a survey have been outlined, 
specific questions have to be constructed. The first two steps are to 
decide on general topic areas and to set the approximate length of the 
survey. The following topic areas were used in one survey developed through 
EMDEC: (1) School-Parent Communications; 

(2) Educational Priorities^ and Problems; 

(3) Neighborhood Schools; 

(4) Staffing Policies; 

(5) Individualized Instruction; 

(6) School Programs; 

(7) Food Service; 

(8) Transportation; and 

(9) Background Information About Respondents. 



* Table II is based on a .95 confidence level. 

IS 
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It is usually necessary to make some difficult decisions regarding 
the length of the questionnaire. That is, should the number of questions 
be increased in order to gain more information, or should the sur-ey be 
abbreviated to increase the probability of a high response rate? In the 
particular case of a mail survey a lengthy questionnaire can significantly 
decrease the rate of response. Within the framework of the length of the 
survey, choices must also be made about the breadth (// major areas) versus 
the depth (// questions per major area) of the questionnaire. It should be 
remembered that future surveys can include new question areas or ask 
additional questions about a particular area. 

Writing the Questions 

The basic guideline for writing questions is to phrase them so that 
the answers received will be meaningful and easily interpreted. The following 
criteria offer more specific suggestions about developing survey questions: 

(1) Avoid negative questions. Asking people if they feel "school 
district funds are spent unwisely" or if "Program A should be improved" will 
bias their answers. Generally, more accurate information can be obtained 
by either asking the question positively, i.e.. "Are school district funds 
spent wisely?" or in a multiple choice framework with shades of difference 
in meaning, i.e.. "Is Program A: (a) very effective, (b) somewhat effective, 
or (c) not effective?" However, there are exceptions to any general rule: 
for example, one school district has asked parents. "Are you getting enough 
information about the schools?" This district expects to get a significant 
number of "no" answers but they followed up with the question. "What kind 
of additional information would you like? With this information the school 
district will be able to improve their communication efforts. 

13 
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(2) Don't assume knowledge. People may not know about a particular 
school district program. A few explanatory sentences may help to clarify 
a program. For example, one school district included this sentence in a 
question about individualized instruction: "Characteristics of individualized 
instruction include frequent assessment of a child's achievement, learning 
activities planned for each child as an individual, and flexible use of 
classroom space." This technique increases the chances that people who answer 
"don't know" to this question either are not familiar with individualized 
instruction or don't have an opinion, rather than merely confused by the term. 
Most people who were not familiar with the term "individualized instruction" 
nay now be able to express their opinions realizing that their children are 
experiencing "individualized instruction" in their educational program. 

(3) Avoid questions which can be answered without surveying. Asking 
people about their child's school attendance, or if they voted in the last 
election, adds unnecessary length to the survey because this information 

is available elsewhere. 

(4) Avoid "iffy" questions. At best, people are poor predictors of 
their own behavior. (Today I feel that I would support a new bond issue, 
but a month from now standing in the voting booth I might not actually pull 
the lever.) At worst, people will try to "game" the question. (Since the 
district is asking me how much I'd pay for my child's lunch program, I'll answer 
40c to influence the price, even though I'd pay 50c for this service.) 

(5) Don't slant questions toward favorable answers. The survey should 
be aimed at getting honest answers from people so the school district can 
better satisfy its constituents. If the survey is designed to make the 
district "look good", the school system is only hurting itself. 

14 
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(6) Don't use the survey as a sales program. Information about the 
value and extent of particular school programs is best transmitted as part of a 
regular communications or public relations program. Including this information 
in a survey will only bias respondent answers to tha questions. The process 

of asking people their opinions about the schools is a very good public 
relations method in itself. 

(7) Don't ask specific policy questions. Asking people about the school 
calendar, for example, may force the school district to either make a poor 
decision or act in opposition to public opinion. While one purpose of a 
survey is to estimate people's opinions about major issues, the survey should 
not nek about specific actiou.- concerning these issues. Decision-makers in 
the school district are in a po...tion where all factors, including cost, must 
be considered. Public opinion should be a major input into decisions, but 

the nchool board and school administration should not use the survey results 
to evade making their own decisions. 

(8) Keep the questions simple. They should not be condescending but 
educational jargon should be avoided. It is easier for most laymen to think 
about a "lunch program" rather than a "food service" or about "reading, 
handwriting, grammar, and spelling" rather than "communications skills." 



15 
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Training Interviewers 



People must be trained for telephone or face-to-face interviewing 
In order to Insure consistent and unbiased results from each interviewer. 
Even if an interviewer has previous survey experience, they will need 
to be trained to conduct the specific survey being used. The following 
guidelines should be used in conducting interviewer training sessions:* 

1. Stress the importance of the interviewer's job. The interviewer 
directly affects the quantity and quality of the information received 
and therefore the value of the survey. Interviewers should know 

how they can affect survey results and why it is important to have 
unbiased results with a high response rate to insure reliability. 

2. Convey items of specific knowledge about the job. Interviewers 
should know how many people they are to survey, when they should con- 
tact these people, how and when to turn In the responses, and 

what to do if a respondent declines the interview. The nature of the 
survey and how the sample was selected should also be reviewed. 
Interviewers will be better motivated and better prepared to do their 
job if they understand the purpose of the survey and the purpose of 
each individual question in the survey. Give the interviewers a 
specific deadline. Ten days to two weeks is a convenient deadline 
once assignments are made. Interviewers must have sufficient time to 
complete their interviews but too much time may result in procrastination 
and Incomplete assignments. 



* A More complete discussion of this topic may be found in Appendix C 
(pp. 574-587), Research Methods in Social Relations (Revised Edition), 
Claire Selltiz, Marie Jahoda, Morton Deutsch, and Stuart Cook, (Holt, 
Rlnehart and Winston), New York, 1959. Also see Chapter 9 (pp. 171-186), 
Survey Research Methods , Earl Babbie, Wadsworth Publishing Co., Belmont, 
California, 1973. -j^Q 



-12- 



Guidellnes about appearance and conduct of interviewers should also 
be stressed. Some suggested guidelines include: 

"Dress to be neat and professional looking but do not overdress 
so as to make the respondent uncomfortable." 

"Don't smoke without first asking if the respondent objects." 

3. Explain the process of conducting an interview. The purpose of 

an interviewer is to create a situation in which the respondent's 

answers will be valid. In order to accomplish this purpose, interviewers 

must establish rapport with the respondent to put the respondent at 

ease, while maintaining a professional manner and securing the desired 

Information. The introduction is a first step in creating the proper 

tone for the interview. One possible introduction is: "Hello, I'm 

for the Area School District. We would 

like to have your opinions about a number of important concerns of 

the schools. You have been selected at random to respond to this 

confidential survey." The interviewer should know the survey well 

enough that the Interview can be carried on in a conversational setting. 

The interviewer should be able to pace the interview so that the 

respondent is neither rushed through the questions nor allowed to digress 

extensively from the questions. Transitional phrases included in the 

survey can assist the orderly completion of the survey. For example, 

"That completes the first section of the questionnaire. The next 

section will concentrate on school district priorities and problem 

areas. First, " Once the survey is completed, the interviewer 

should close the interview without rushing off or overstaying. An 

1? 
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effective means of handling this closing phase of the interview is to 
thank the respondent for their cooperation while making a leave- 
taking gesture such as putting the survey or survey notes in a 
folder or brief case. In the case of a telephone survey less rapport 
would have been developed between the respondent and interviewer, so 
a simple closing statement will suffice. 

4. Prepare interviewers with quick and uniform responses to certain 
questions or situations. These likely events include such respondent 
remarks as: 

"Why was I selected?" 

"How do I know you represent the school district?" 

"I'm too busy to answer." 

"How long will this take?" 

"What's this survey aboat?" 

"Please repeat the question." 

"What do you mean by (some term) ' ?" 

Standard responses for these situations have been developed and are 
reviewed in many interviewing guides. These uniform methods of 
Interviewer replies to such situations will usually result in the 
successful completion of the questionnaire. It is important that each 
interviewer know how much latitude they have in replying to respondent 
questions or uncertainty. "What do you mean by individualized 
instruction (or some other term)?" is an example of a common situation. 
Interviewers may be instructed to reply by only repeating the question 
or by trying to explain the term. The best method, generally, is to have 
a set of common definitions available to be used by each interviewer 

In such a situation. . ^ 

18 
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5. Have the interviewers participate in at least one practice 
interview and observe at least one more. This increases the 
interviewer's confidence and offers an opportunity for observing the 
Interviewers before they are conducting actual, field Interviews. One 
method is to have the interviewers Initially practice on each other. 
Then they may conduct a practice interview with one of the trainers 
who will role play and introduce some of the typical situations (see 
guideline 4, above) confronting interviewers. Videotaping is an 
especially effective means for reviewing an Interviewer's practice 
Interviews. In addition to training for standardization of 
questionnlng, training should also cover standardization of response 
recording. One method for this training is to have a number of 
interviewers simultaneously observing a staged interview and independently 
recording the responses. After each question, responses can be 
compared and issues discussed so that interviewers will be using uniform 
methods of recording responses when conducting interviews. 

6. Have each interviewer comment on the survey. Comments about the 
ease or difficulty of getting responses to each question and sugges- 
tions for improving the survey will be extremely useful in designing 
new surveys. Interviewers should record these comments immediately 
after each Interview or at the end of the last interview of a day if 
several are conducted back-to-back. However, these comments should 

never be recorded during an Interview. (This would most likely interfere 
with the interviewer's concentration on the interview.) 

7. Inform interviewers of the validation procedure. Validation 
involves having the school district (or whoever is conducting the 
survey) call up a portion of each interviewer's respondents to find 
out how the interview went. This procedure is also a good communica- 

13 
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tion method with the public and can offer an opportunity for 
answering questions about the survey. The validation need not 
threaten interviewers as most comments about the interviews will be 
favorable and can serve as a basis for complimenting the interviewers 
Mhea their jobs are completed. 

Pre-testing the Survey 

As Robert Bums observed, "the best laid plans of mice and men often 
go astray." Pre-testing is an important way to iron out problems before 
the actual survey. Administering the survey to 20 or more people who are 
similar to the sample selected is very strongly recommended. While the 
survey should be as thoroughly developed as possible before pre-testing, 
pre-testing provides the acid test, "Do people understand the questions 
and can they make meaningful responses?" Changes made after a pre-test, 
often simple ones, can avoid wasting time and effort. For example, 
one school district asked parents to rate the importance of each of Pennsyl- 
vania's Ten Goals of Quality Education on a scale (from 1 to 5) . A 
pre-test showed that people generally rated each goal near the very 
important end of the scale. Since this question didn't differentiate 
people's opinions about these 10 goals, the question was revised to ask 
parents to rank order the 10 goals. This should provide more meaningful 
Information about people's opinions. 

Analyzing the Results 

Survey results may be analyzed and described by the following means : 
1. Tabulation of number and percentage of people responding to 
each part of each question. For example, 
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Questlon 29; Do you feel that sex discrimination exists in the schools? 

Responses % p 

YES 30% 120 

NO 60% 240 

DON'T KNOW 10% 40 



TOTALS 100% 400 

2. Tabulation of responses for different groups of people (e.g., male 
and female, School A parents and School B parents). For example. 
Question 29 ; Do you feel that sex discrimination exists in the schools? 

Responses Franklin School Jefferson Elementary 

YES 36Z 24% 

NO 54% 66% 

DON'T KNOW 10% 10% 

3. Cross-tabulations of responses for two questions. For example. 

Question 3; Are you getting enough information about the schools? 

Question 12; Do you feel that the school district central office administra- 
tion is generally responsive to what parents have to say 
concerning the schools? 



Question 3 



Question 12 
# of Respondents Who Sav; 




YES 


NO 


TOTALS ■ 


240 (60Z) 


160 (40%) 


YES 140 (35%) 


100 


40 


NO 260 (65%) 


140* 


120 



*These 140 people answered NO to Question 3 and answered YES to Question 12 
The other elements in this table can be similarly interpreted 
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In our example, the cross-tabulation shows that people answering yes 
to Question 3 were more likely, than those who answered no, to answer yes 
to Question 12. 

4. Inferences about causal relations may be drawn from any of the 
tabulations. From the previous examples, these inferences might be 
drawn: 

a) Since Jefferson Elementary has a female principal, people are 
less likely to feel that sex discrimination exists in the school. 

b) People's attitudes about the schools are primarily based on 
personal experiences. Therefore, if the central office administra- 
tion makes a point of being responsive to parents or if school- 
parent communications are improved, parent attitudes about the 
schools will be more positive in general. 

It should be cautioned that inferences are just that. However, some 
of them may be tested and they can provide the basis for planning future 
action. For example, a school district could plan an Improved school- 
parent communications program. One evaluation of this Improvement and a 
corroboration of inference (b) above, would be to re-survey a sample of 
the school district's parents. If the program is successful and the interence 
Is correct, improved school-parent communications will be reflected in 
more positive attitudes about school administrators, in addition to more 
positive attitudes about information from the schools. 

Reporting the Results 

Usually a school district conducting a survey will announce their 
plans in school newsletters or local newspapers. A general announcement 
can generate interest in the survey and Increase the willingness of people to 
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participate. Agreeing to release the survey results will usually stimulate 
participation and is a positive step in improving school-community relations. 

Survey results reported by the district usually include a simple 
tabulation of responses to each question and a summary. If the school 
board or administration have reviewed the results and have made plans to 
follow the survey with action in particular areas, these can be mentioned. 

Survey results can be used for the attention of specific groups. For 
example, attitudes about the responsiveness of school building administra- 
tors may be publically reported for the district as a whole, but not for 
particular school buildings. The response to this question for specific 
buildings is important information for the superintendent but might be 
harmful if reported to the public. Individual judgements will have to be 
made for cases like this. A general rule is that the disclosure of informa- 
tion to the community is beneficial and can provide a valid basis for 
assessing progress made by the school district in certain areas. But 
reporting information is inadvisable if it reflects the performance of 
any single individual in the school district. The distinction can be made 
that the community should have knowledge about program assessments but 
Information about the assessment of individual performance should be given 
only to those people with a direct need for such information. 

Community response to the reporting of the survey results can be useful 
in developing the format and types of information to be used after conducting 
future surveys. 

Summary 

How people^ feel about a school system and its programs is important 
Information for school board memb^s and administrators. Opinion surveys, 
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if targeted to ask specific questions of a particular group (or groups) , 
can be an effective means of getting such information. The major concerns 
in conducting a valid survey are; 

(a) to get a representative sample of opinion; and 

(b) to obtain unbiased expressions of people's opinions. 
Methods exist to facilita-.e the completion of valid opinion surveys. 

Once the survey information is collected it can be reported to 
various groups in a variety of ways. The survey results can consist of 
everything from a straightforward tabulation of total responses to each 
question to a detailed analysis of how different groups (men, women, people 
under 35 years of age, people with three or more children in school, etc.) 
responded to particular questions. These results should provide a signifi- 
cant input to future school district decisions. In addition, if surveys 
are conducted periodically, changes in opinion trends can be tracked over 
several years. 
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